131 research outputs found
Beyond single-shot text queries: bridging the gap(s) between research communities
This workshop brings together researchers from different streams and communities that deal with information access in the widest sense. The general goal is to foster collaboration between the different communities and to showcase research that sits at the border between different areas of research
Is ChatGPT a Biomedical Expert? -- Exploring the Zero-Shot Performance of Current GPT Models in Biomedical Tasks
We assessed the performance of commercial Large Language Models (LLMs)
GPT-3.5-Turbo and GPT-4 on tasks from the 2023 BioASQ challenge. In Task 11b
Phase B, which is focused on answer generation, both models demonstrated
competitive abilities with leading systems. Remarkably, they achieved this with
simple zero-shot learning, grounded with relevant snippets. Even without
relevant snippets, their performance was decent, though not on par with the
best systems. Interestingly, the older and cheaper GPT-3.5-Turbo system was
able to compete with GPT-4 in the grounded Q&A setting on factoid and list
answers. In Task 11b Phase A, focusing on retrieval, query expansion through
zero-shot learning improved performance, but the models fell short compared to
other systems. The code needed to rerun these experiments is available through
GitHub.Comment: Preprint accepted at the 11th BioASQ Workshop at the 14th Conference
and Labs of the Evaluation Forum (CLEF) 202
Third International Workshop on Gamification for Information Retrieval (GamifIR'16)
Stronger engagement and greater participation is often crucial
to reach a goal or to solve an issue. Issues like the emerging
employee engagement crisis, insufficient knowledge sharing,
and chronic procrastination. In many cases we need and
search for tools to beat procrastination or to change people’s
habits. Gamification is the approach to learn from often fun,
creative and engaging games. In principle, it is about understanding
games and applying game design elements in a
non-gaming environments. This offers possibilities for wide
area improvements. For example more accurate work, better
retention rates and more cost effective solutions by relating
motivations for participating as more intrinsic than conventional
methods. In the context of Information Retrieval (IR)
it is not hard to imagine that many tasks could benefit from
gamification techniques. Besides several manual annotation
tasks of data sets for IR research, user participation is important
in order to gather implicit or even explicit feedback
to feed the algorithms. Gamification, however, comes with
its own challenges and its adoption in IR is still in its infancy.
Given the enormous response to the first and second
GamifIR workshops that were both co-located with ECIR,
and the broad range of topics discussed, we now organized
the third workshop at SIGIR 2016 to address a range of
emerging challenges and opportunities
Combining Minimally-supervised Methods for Arabic Named Entity Recognition.
Supervised methods can achieve high performance on NLP tasks, such as Named Entity Recognition (NER), but new annotations are required for every new domain and/or genre change. This has motivated research in minimally supervised methods such as semi-supervised learning and distant learning, but neither technique has yet achieved performance levels comparable to those of supervised methods. Semi-supervised methods tend to have very high precision but comparatively low recall, whereas distant learning tends to achieve higher recall but lower precision. This complementarity suggests that better results may be obtained by combining the two types of minimally supervised methods. In this paper we present a novel approach to Arabic NER using a combination of semi-supervised and distant learning techniques. We trained a semi-supervised NER classifier and another one using distant learning techniques, and then combined them using a variety of classifier combination schemes, including the Bayesian Classifier Combination (BCC) procedure recently proposed for sentiment analysis. According to our results, the BCC model leads to an increase in performance of 8 percentage points over the best base classifiers
Motivations for Participation in Socially Networked Collective Intelligence Systems
One of the most significant challenges facing systems of collective
intelligence is how to encourage participation on the scale required to produce
high quality data. This paper details ongoing work with Phrase Detectives, an
online game-with-a-purpose deployed on Facebook, and investigates user
motivations for participation in social network gaming where the wisdom of
crowds produces useful data.Comment: Presented at Collective Intelligence conference, 2012
(arXiv:1204.2991
Beyond single-shot text queries: bridging the gap(s) between research communities
This workshop brings together researchers from different streams and communities that deal with information access in the widest sense. The general goal is to foster collaboration between the different communities and to showcase research that sits at the border between different areas of research
Interactive query expansion for professional search applications
Knowledge workers (such as healthcare information professionals, patent agents and recruitment professionals) undertake work tasks where search forms a core part of their duties. In these instances, the search task is often complex and time-consuming and requires specialist expert knowledge to formulate accurate search strategies. Interactive features such as query expansion can play a key role in supporting these tasks. However, generating query suggestions within a professional search context requires that consideration be given to the specialist, structured nature of the search strategies they employ. In this paper, we investigate a variety of query expansion methods applied to a collection of Boolean search strategies used in a variety of real-world professional search tasks. The results demonstrate the utility of context-free distributional language models and the value of using linguistic cues to optimise the balance between precision and recall
GamifIR 2016: SIGIR 2016 Workshop on Gamification for Information Retrieval
The third workshop on Gamification for Information Retrieval (GamifIR) took place on the 21th of July 2016 in conjunction with SIGIR 2016 in Pisa, Italy. It was the first GamifIR held in conjunction with the SIGIR, the first and second GamifIR workshops were both colocated with ECIR. The workshop program included one invited keynote presentation, seven paper presentations and a discussion session. The keynote presentation stated the necessity of proper theory for gamification design and resulting opportunities. The paper presentation covered studies on diverse areas and approaches for the application of gamification
- …